These pictures are usually produced using a so-called "interlaced scan" standard: each picture frame is split into two successive fields, which are offset in space by one half line in the vertical direction and which are offset in time by the time required to scan one field.
Interlacing suffers from several drawbacks. It considerably complicates: converting pictures from one television standard to another; space-time analysis of pictures; and also improving pictures by applying image processing techniques thereto. Image information has a three dimensional structure (two space dimensions, plus time) and when scanning is interlaced this structure is not orthogonal: a given point in a field has no equivalent in the following field; with the following field only containing points that are adjacent to the given point, with said adjacent points being absent from the field containing the given point.
It is therefore necessary to perform "deinterlacing", i.e. to convert from interlaced scanning using two successive fields to sequential scanning in which successive frames are identical in structure and superposable.
Deinterlacing makes use of interpolation between image points in space-time. Each image point or "pixel" is defined by two space co-ordinates (x along horizontal lines and y in a vertical direction perpendicular to the lines) together with a time co-ordinate (t) associated with the interlaced field or non-interlaced frame to which the pixel belongs.
There are numerous studies concerning deinterlacing, and the following may be mentioned:
J. Santillana, B. Choquet, "Algorithmes de detection et d'estimation de mouvement dans des images de television" (Algorithms for detecting and estimating motion in television pictures), CCETT: RTI/T/007/85/JS-BC, March 85. B. Choquet, "Inventaire des techniques de traitement d'images en conversion de normes de representation pour la television Haute Definition" (List of image processing techniques for conversion between picture standards in HDTV), CCETT: RTI/NT/031/86/BC, Nov. 25, 1986.
D. Pele, B. Choquet, "Algorithmes d'analyse spatiotemporelle pour Television Haute Definition" (Algorithms for space-time analysis in HDTV), CCETT: CAL/T/002/87/DP and RTI/T/004/87/BC, Apr. 87.
B. Choquet, P. Siohan, A. Benslimane, "Detection d'activite, dans une sequence d'images, par gradient temporel et par gradient spatial" (Detecting activity in a sequence of images by means of a time gradient and a space gradient), CCET: RTI/T/017/85 BC and CAL/T/006/85/PS.AB, July 85.
P. Siohan, B. Choquet, "Multidimensional processing for motion detection," PCS 86: Tokyo, Apr. 2-4, pages 65, 66, session 3.18.
B. Choquet, P. Siohan, "Enhancement techniques of a motion detector in High Definition Television," IEE, Second International Conference on Image processing and its applications, Conference publication number 265, June 24-26, 1986 (pages 215-219).
P. Siohan, B. Choquet, "Motion detection for the deinterlace in HDTV picture processing," International Eurasip Workshop on Coding of HDTV: Nov. 12-13, 1986, l'Aquila, Italy (Vol. 2).
B. Choquet, P. Siohan, "Desentrelacement par interpolations adaptatives" (Deinterlacing by adaptive interpolation), Colloquim on HDTV, Ottawa, Oct. 4-8, 1986.
D. Pele, "Quelques methodes de segmentation d'images" (Some image segmentation methods), ENST-Bretagne: 86/LEMPAB/RCM/01.
In spite of these numerous studies, there is no deinterlace technique available at present which is simultaneously simple, fast, and effective, in particular for high definition television (HDTV) for which the image quality required in base band means that the techniques used in the past need to be thoroughly overhauled.
The criterion of simplicity relates to the number of data items that need to be taken into account when performing deinterlacing. Proposals have been made, for example, to use a lowpass linear filter whose input information is constituted by 11 frame lines (vertical aperture) over 7 fields (time aperture). It is clear that such a lowpass linear filter is at the limit of acceptable complexity.
The speed criterion relates to the speed at which the means used can act: naturally it is necessary for them to be capable of operating in real time, i.e. at least as fast as the rate at which picture information is renewed. This leads to a compromise between the amount of input information which can be taken into account and the sophistication of the means which process said information. For example, in the above-mentioned linear filter, it is partically impossible to change the interpolation parameters as a function of the local contents of a picture.
The effectiveness criterion relates to the results obtained on deinterlaced pictures. In particular, the following defects must be avoided:
loss of definition in non-moving zones of the picture, which phenomenon is particularly critical and unacceptable for an observer;
loss of definition in zones which are in motion, which becomes unacceptable whenever the eye is following the moving object;
diagonal outlines which are degraded since they are reproduced in "staircase" form; and
poor reproduction or total loss of outlines which are moving quickly or very quickly.
The object of the present invention is to provide an improved method and apparatus for processing image signals, and suitable, in particular, for substantially improving the present situation on deinterlacing techniques.
The present invention is based on a method comprising the following stages:
(a) selecting a time pair and a space pair of main points about a desired point in a line to be interpolated in a field being interpolated, the main points of said time pair having the same position in the image as the desired point, but being situated in the two fields occuring immediately before and immediately after the field being interpolated, while the main points of the space pair come from the field being interpolated, from respective ones of the two positions which are vertically adjacent to the position of the desired point;
(b) calculating the "time" main difference between the signals at the two points in the time pair, and also the "space" main difference between the signals at the two points in the space pair;
(c) selecting an interpolation function taking account of said main differences; and
(d) calculating the interpolated signal of the desired point by means of said interpolation function.
In other words, it is known to take video input signals for the purpose firstly of detecting motion in a zone surrounding a point to be interpolated and secondly for selecting an interpolation function from a library of such functions. These two types of information are used for deciding on the choice of a given algorithm and for establishing transition criteria between two different algorithms. This gives rise to a final interpolation function which is applied to the input signals or to signals derived therefrom.
For example, motion may be detected on the basis of the time main difference and the space main difference, for which various proposals have already been made in the prior art.